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Abstract — In this paper we explore the use of electrical 
biosignals measured on scalp and corresponding to mental re- 
laxation and concentration tasks in order to control an object in 
a video game. To evaluate the requirements of such a system in 
terms of sensors and signal processing we compare two designs. 
The first one uses only one scalp electroencephalographic (EEG) 
electrode and the power in the alpha frequency band. The 
second one uses sixteen scalp EEG electrodes and machine- 
learning methods. The role of muscular activity is also evaluated 
using five electrodes positioned on the face and the neck. Results 
show that the first design enabled 70% of the participants to 
successfully control the game, whereas 100% of the participants 
managed to do it with the second design based on machine 
learning. Subjective questionnaires confirm these results: users 
globally felt to have control in both designs, with an increased 
feeling of control in the second one. Offline analysis of face 
and neck muscle activity shows that this activity could also be 
used to distinguish between relaxation and concentration tasks. 
Results suggest that the combination of muscular and brain 
activity could improve performance of this kind of system. They 
also suggest that muscular activity has probably been recorded 
by EEG electrodes. 

I. INTRODUCTION 

In recent years, researches have been conducted to use 
electrical brain activity for interacting with video games and 
virtual environments [?], [?], [?]. In this context, researchers 
have started to explore signals correlated to concentration 
and relaxation states [?], [?], [?]. Several electroencephalo- 
graphic (EEG) markers have indeed been identified as corre- 
lated with mental relaxation, task engagement, attention and 
mental workload. For example, alpha rhythm (8Hz-12Hz) 
is known to be attenuated or blocked by mental activity 
and attention [?]. Other rhythmic activities such as beta 
and theta rhythms have been investigated to quantify mental 
relaxation [?] and attention level [?]. Hamadicharef et al. [?] 
have also assessed the level of attention through EEG, 
but employing machine learning methods to learn the best 
frequency and spatial filters to use. This approach reveals 
better classification accuracy than using only alpha, beta and 
theta bands. More recently Miihl et al. proposed to use alpha 
activity as a passive Brain Computer Interface (BCI) modal- 
ity [?]. Alpha activity was used in an implicit way to adapt 
the game content. Enemies speed was increased when the 
alpha activity was low (which was expected to be correlated 
with a low relaxation state). Unfortunately, no significant 
differences for game experience and for user's alpha power 
modification were found. Electrical activity has also been 
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used in an explicit way. In the game BrainBall [?] (later 
adapted into a commercial game called MindbalU) players 
must be relaxed to move a ball away. A ratio between beta 
and alpha activity measured with forehead electrodes was 
used to control the ball position. Matel Inc. also proposed 
the MindFlex garnet in which the player has to concentrate 
to control physical objects. However, we do not have infor- 
mation about the electrical signal and the signal processing 
used in these games. Despite these promising first results, 
there is still a number of important scientific questions about 
such systems that need to be answered. In particular: 

• Quantitative and in depth performance analysis are 
scarce in the litterature; 

• Due to a lack of comparative study, the best elec- 
trodes (number and placement) and signal processing 
to be used are still not well known; 

• There is still a debate about whether such systems 
are pure EEG systems, independent of Electromyogram 
(EMG) activity, or if EMG plays an important role [?]. 

Thus, in this paper we have conducted a study in which users 
had to perform mental concentration and relaxation tasks to 
control an object in a video game, based on EEG recordings, 
which aims at addressing the aforementioned questions. 

The remainder of this paper is organized as follows. In 
Section [Til the application developed and the experimental 
procedure are described. In Section [Till results are presented 
and discussed. Finally, the main conclusions are summarized 
in Section [IV] 

II. METHOD 

A. Application 

The goal of the game was to control the altitude of a 
spaceship displayed on the screen. Two kinds of instructions 
were provided to the user: "Go Up" and "Go Down". When 
"Go Up" instruction was displayed, the user had to try to 
make the spaceship go up and stay at a high altitude during 
35 s. The opposite behavior was expected for the "Go Down" 
instruction, by making it go down. The spaceship altitude 
depended on the user's level of concentration or relaxation 
as computed by the system (see section Ill-El l. The maximum 
and minimum altitudes were respectively 1 and —1. A green 
line that symbolizes the smoke of the spaceship was used to 
provide visual feedback to the participants. Figure Q] displays 
a screenshot of the application during typical "Go Down" and 
"Go Up" phases. 



http : / /www . mindball . se /product . htmlj 



http : //mindf lexgames . com/ 




Fig. 1 . The video game application with feedback during the two different 
phases (Go Down on the left, Go Up on the right). 

B. Procedure 

The experiment was carried out in an office room. Par- 
ticipants sat in front of a 24" LCD screen device. The 
distance to the screen was 90 cm. Before the experiment 
began, all participants were informed about the aim of the 
study and the relation between the spaceship position and 
their mental state; and they filled out a consent form. To 
make the spaceship go up, they were suggested to follow 
one of the two strategies: "focus your attention on the 
spaceship altitude" and "imagine sending mental orders 
to the spaceship such as Go up!". To make the spaceship 
go down, they were suggested to follow one of the two 
strategies: "do not focus your attention on the spaceship and 
its altitude" and "focus your attention on your respiration 
patterns to relax". Participants were instructed to look at 
the display screen during both conditions. They were also 
informed that the recorded signals are sensitive to artifacts 
(like body movement, teeth clenching), for these reason they 
should move as little as possible throughout the experiment. 

The experiment consisted of two parts (Part A and Part B) 
corresponding to two setups (a version based on alpha 
activity and a version based on machine learning methods, 
respectively). To assess differences between the two setups, 
the experiment had a within-subjects design. Thus all the par- 
ticipants performed each part of the experiment on a different 
day, separated by less than a week. To eliminate possible 
order effects between versions, half of the participants started 
with the first version (group A did Part A first) and the other 
half with the second version (group B did Part B first). 

Each part of the experiment consisted of six sessions. Each 
session was divided into three blocks. Each block had two 
phases: one "Go up" and one "Go Down", starting with 
a randomized order. Phase duration was 35 s. At the start 
of each game, the instructions were displayed at the top 
of the application screen for 5 s ("Go up" or "Go down"). 
The spaceship was displayed and its altitude was directly 
mapped to the control signal (see section III-Eb . The blocks 
were separated by a break of 10 s, during this time the screen 
was blanked. Each session was separated by a break of less 
than 2 min. This time was used to run the machine learning 
process between sessions (see section III-EI >. 

C. Participants 

Ten participants (8 men, 2 women) aged from 22 to 34 
(/i = 26.2, cr = 4) took part in the experiment. Subjects were 
randomly assigned to the two different groups A and B. 



D. Electrodes montage 

Two sets of electrodes were used to record data throughout 
the experiment. The first set of electrodes (EEG set) was used 
to record EEG activity. The purpose of the second set (EMG 
set) was to record EMG activity on the face and the neck. A 
Gtec UsbAmp was used to acquire EEG data at a sampling 
rate of 512 Hz (EEG set). This device enabled to use 16 
electrodes according to the 10-20 international system. It was 
used to record EEG activity related to mental relaxation and 
concentration tasks. The electrodes used were: Fpl, Fpz, Fp2, 
F3, Fz, F4, T7, C3, Cz, C4, T8, P3, Pz, P4, Ol and 02. 
Electrodes Fpl, Fp2, Pz were chosen accordingly to BCI 
literature [?], [?]. The remaining 11 electrodes were placed 
on a grid pattern to cover an important surface of the scalp. 
A ground electrode (located on the left ear) and a reference 
electrode (located at AFz position) were also used. 

To record electrophysiological activity on the user face 
and neck a MindMedia NeXus 32b device was used (EMG 
set). Four electrodes were placed on the face (above and 
below each eye), and one electrode was placed on the neck. 
A ground electrode (located on the nose) and a reference 
electrode (located on the forehead) were also used with this 
setup. These electrodes were not used for online control of 
the game but only for offline EMG analysis. 

E. Signal Processing 

Signal acquisition and online processing were conducted 
using the open-source software OpenViBE [?]. Offline pro- 
cessing was done using Matlab. Two control signals were 
used during the online experiment. The first one was com- 
puted using only 1 scalp EEG electrode and power in the 
alpha frequency band (CS1). The second one was acquired 
from 16 scalp EEG electrodes and processed by applying 
machine learning methods (CS2). 

The first control signal (CS1) corresponds to the opposite 
of the power of the user's alpha frequency band at Pz 
location. Indeed alpha activity is known to increase when 
the user is relaxed [?]. As this activity is mainly found 
in the posterior half of the head [?], the electrode was 
placed on Pz position. This activity was quantified by a band 
power technique [?]. Band power was computed 16 times per 
second on a moving window of 1 s. The alpha band selection 
was based on the method of individually adjusted bandwidths 
(IBFW) proposed by Doppelmayr et al. [?]. To determine 
the individual dominant alpha frequency of each participant, 
we used a 30 s recording of the EEG data with participants 
closing their eyes. Then, the spectrum of the signal was 
computed and the frequency (in 6 Hz-20 Hz) that shows the 
maximum power was chosen. A fixed width of 4 Hz around 
this value was then used as the subject's alpha frequency 
band. A calibration phase with no feedback was used to scale 
the control signal between —1 and 1. The calibration phase 
corresponded to a 30 s relaxation task and 30 s concentration 
task, both without any feedback. 

To compute the second control signal (CS2), a technique 
similar to the one proposed in [?] was used. All EEG 
signals were passed through a bank of 4 Hz bandwidth 



TABLE I 

Questionnaire marks reported (mean±std). 



Question 



PartA(CSl) Part B (CS2) 



Did you feel in control of the 

spaceship during "Go Up"? 

(1: never, 7: always) 

Did you feel in control of the 

spaceship during "Go Down"? 

(1: never, 7: always) 

Did you feel fatigue related to the 

interaction? (1: very, 7: not at all) 

How fun was the game? 

(1: boring, 7: enjoying) 



4.4 ±1.51 4.9 ±1.29 

4.4 ± 1.07 5.45 ±1.26 

4.3 ± 1.25 4.9 ±1.20 

3.8 ± 1.67 4.2 ±1.48 



The first column corresponds to the use of alpha band power technique with 
one electrode Pz (CS1) during part A. The second column corresponds to 
the use of 16 electrodes and machine-learning method (CS2) during part B. 



filters centered on all the frequencies between 2 Hz and 
30 Hz. A common spatial pattern (CSP) method [?] was then 
used to compute spatial filters for each of them. Maximum 
relevance feature selection was used to select the five most 
relevant couples of frequency band and spatial filter [?]. The 
linear model that best matched features (computed using 
the selected band pass filters and spatial filters) and both 
classes (Up and Down) was then determined using linear 
regression on the training set. The linear equation, selected 
band filters, and selected spatial filters were then used online. 
The learning process was run between each session using all 
data recorded during the previous sessions of the day. 

In part A, CS1 was used during the whole experiment 
(altitude of the spaceship was mapped to CS1). In part B, 
CS 1 was used for the first session to provide feedback to the 
user when collecting training data for the subsequent CS2 
calibration (machine learning); CS2 was used during all the 
remaining sessions. 

III. RESULTS 

A. Subjective questionnaire 

After each part of the experiment, participants were asked 
to fill in a questionnaire. Table U provides the average marks 
they gave for each question. Results showed that participants 
felt that they had control over the spaceship in both parts. 
Indeed, the reported sensation of control (average marks 
given for question 1 and 2) was on average 4.4 for Part A 
and 5.17 for Part B on a scale between 1 (never) and 7 
(always). During Part A, there was no significant difference 
between the "Go up" and "Go down" conditions (Wilcoxon 
paired test p = .95). Concerning part B, there seemed to 
be a difference between "Go up" and "Go down" condition, 
but it was not significant (Wilcoxon paired test p = .35). 
Concerning the influence of the control signal, a significantly 
better feeling of control was observed in Part B (Wilcoxon 
paired test p = .02). 

Questionnaire results about tiredness seemed to indicate 
that participants felt the experiment relatively tiring. Part B 
was reported to be less tiring, although this difference was 
not statistically significant (Wilcoxon paired test p — .20). 
Participants were also asked to report when they started to 
feel fatigue during the experiment. Fatigue seems to start 



after the middle of the experiments (54% completed for 
Part A, and 62 % completed for Part B). In the middle of 
the experiment, participants have already completed 9min 
of active control ("Go up" and "Go down"). 

Concerning the fun, it seems that participants felt an 
average enjoyment during the game. This could be due to the 
simplicity of the interface and the absence of rewards during 
the game. No significant difference was found between the 
two parts (Wilcoxon paired test p = .28). 

B. Online Performances 

The first 5 s of each phase were ignored for the analysis of 
the online performances (it corresponded to the instruction 
reading time). We assessed performances by computing the 
average difference between the altitude in the two conditions 
("Go up", "Go down") over each block. Indeed, a higher 
altitude in the "Go up" condition could be interpreted as a 
success. The online performance indexes are provided in Ta- 
ble|nl Spearmann correlation test between performance index 
and participant's control marks indicates that the difference 
between conditions seems to be correlated to reported marks 
(Part A r = .56, p = .09; Part Br = .72, p = .02). 

The computation of a Student t-test, between conditions 
"Go up" and "Go down" in each block shows that there is a 
significant difference between the two conditions for seven 
out of the ten participants with CS1 (Part A), and for all 
the participants with CS2 (Part B) (asterisks in Table Hi] ). 
When the difference is significant it is also always positive: 
the spaceship altitude in the "Go up" condition is higher 
than during the "Go Down" condition. Regarding to the 
influence of the control signal it appears that the use of CS2 
with 16 EEG electrodes provided better overall performances 
compared to CS1 (Wilcoxon paired test p = .002). It 
highlights the need of machine learning and subject specific 
design and confirms results in [?]. 

TABLE II 

Performances expressed as difference between spaceship 

ALTITUDE IN EACH CONDITION (Z up — Z domn ). 





Online 


Offline 


N. 


Part A (CS1) 


Part B (CS2) 


Part B (CS3) 


Part B (CS4) 




EEG 


EEG 


EMG 


EEG + EMG 


1 


-0.07 (0.24) 


0.22 (0.30)** 


0.24 (0.16)*** 


0.42 (0.34)*** 


2 


0.18 (0.28)* 


1.03(0.21)*** 


0.55 (0.22)*** 


1.32 (0.16)*** 


3 


0.30 (0.25)** 


0.95 (0.28)*** 


0.95 (0.32)*** 


1.14 (0.29)*** 


4 


0.32 (0.16)*** 


0.42 (0.25)*** 


0.37 (0.14)*** 


0.56 (0.35)*** 


5 


-0.05 (0.13) 


0.24 (0.31)* 


0.30 (0.17)*** 


0.79 (0.29)*** 


6 


0.22(0.11)*** 


0.91(0.16)*** 


0.17 (0.23)* 


0.92 (0.24)*** 


7 


0.05 (0.26) 


1.02 (0.39)*** 


0.35 (0.19)*** 


1.06 (0.42)*** 


8 


0.31(0.13)*** 


0.35 (0.29)** 


0.17 (0.20)** 


0.44 (0.38)*** 


9 


0.20 (0.14)** 


0.56 (0.29)*** 


0.15 (0.13)*** 


0.51 (0.25)*** 


10 


0.13 (0.19)* 


0.57 (0.33)*** 


0.31 (0.22)*** 


0.56 (0.32)*** 


M 


0.16 


0.63 


0.36 


0.77 



The first column corresponds to the use of alpha band power with electrode 
Pz (CS1). The second column corresponds to the use of machine-learning 
methods with 16 EEG electrodes (CS2). Offline performances using EMG 
set are presented in column 3. Offline performances using the combination of 
EEG and EMG sets are provided in column 4. Last line provides the average 
altitude difference over participants. Values are presented as mean (std); 
asterisks represent the significance of the difference (*: significant at p < 
.05; **: significant at p < .01; ***: significant at p < .001) 
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Fig. 2. Mean altitude of the spaceship on screen as function of different 
conditions ("Go-Up" vs. "Down" and CS1 vs. CS2). Altitude was averaged 
over all blocks and all participants. Dotted line represents altitude when 
alpha frequency band at Pz electrode was used (CS1). Solid line represents 
altitude when machine-learning method was used with 16 electrodes (CS2). 

The analysis of performances evolution over time is shown 
in Figure |2] It provides the average evolution of altitude for 
both conditions and both control signals. We observed an 
increase of performance over the course phase duration with 
CS2. This suggests that users need time to get into the task. 

C. Role ofEMG 

Co-occurrence of muscular activity is known to result 
in contamination of EEG by Electromyogram (EMG) [?]. 
Therefore an evaluation of EMG activity during the interac- 
tion is necessary. For this purpose signals recorded during 
Part B using face and neck electrodes (EMG set) were 
analyzed. We computed a control signal (CS3) based on 
EMG set employing the same machine learning methods 
as the one used for CS2 (see section III-Eb . The usage of 
EMG set led to a significant difference between the "Go 
Up" and "Go Down" conditions for all the participants 
(Table Hill. This means that the recorded information in facial 
and neck electrodes can be used to discriminate between the 
two conditions although the use of EEG set (CS2) provided 
better performances (Wilcoxon paired test p — .033). One 
interpretation of these results is that EMG set measured 
some EEG activity, which is enough to discriminate between 
tasks (indeed electrodes above the eyes are able to measure 
some frontal EEG activity). A more likely interpretation 
is that EMG set measured EMG activity, which provides 
information to discriminate between tasks. This would mean 
that a discriminant muscular activity occurred when users try 
to do the two mental tasks, however they were asked to try to 
avoid muscular activity during the experiment. This second 
interpretation would also imply that the EEG set measured 
a part of this discriminant muscular activity. 

To evaluate the combination of EEG and EMG sets in 
terms of performance the computation of a control signal 
(CS4) based on the three most relevant EEG features and 
the two most relevant EMG features was done (to have 5 
features as with EEG or EMG alone). The combination of 
EMG and EEG allowed to increase performances in terms of 
difference between conditions compared to CS2 (Wilcoxon 
paired test p = .019). This result may indicate that there are 



relevant information in the two sets of electrodes that are not 
redundant between the sets. Future work should be done to 
quantify the part of information in each set and to qualify the 
origin of this information (muscular versus brain activity). 

IV. CONCLUSION 

In this paper, we have explored the use of two different 
setups based on scalp electrical biosignals corresponding to 
mental relaxation and concentration tasks to control a video 
game. Results of this experiment are surprisingly good and 
suggest that both setups seem well suited for interaction with 
simple video games. Analysis of signal from face and neck 
electrodes (EMG set) suggests also that there is relevant in- 
formation in muscular activity correlated to the two different 
tasks and that the combination of EEG and EMG electrodes 
in a hybrid BCI [?] paradigm could improve performance. 
They also suggest that the current BCI approaches using EEG 
to detect concentration and relaxation tasks (e.g. [?]) may not 
be pure BCI as they may be involuntarily using EMG signals 
recorded with EEG electrodes. 
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